1.
Introduction

Figure 1 shows a slab of tomb biography of the Tang dynasty.


¹

Researchers can copy the words on such slabs to produce a collection of tomb biographies for research. A typical tomb biography contains various types of information about the deceased and their families and, sometimes, a rhyming passage of admiration. Employing software tools to analyze the texts, one can extract useful information from the collections of tomb biographies to enrich databases like the China Biographical Database (CBDB) to support further Chinese studies.


²

Figure 1. A slab of tomb biography of the Tang dynasty

It is well known that modern Chinese texts do not include delimiters like spaces to separate words. Hence, researchers design algorithms for segmenting Chinese character strings into words (Sun et al., 2004; Shao et al., 2017).

In contrast, it is not as well known that, in classical Chinese, there were no markers for the separation of sentences. The characters in Figure 1 simply connect to each other. In modern Chinese, texts are punctuated for pauses in sentences and ends of sentences. The research about algorithmically inserting these syntactic markers into classical Chinese is receiving more attention along with the growth of digital humanities in recent years. The needs of segmenting ancient texts for humanities studies are not unique to Chinese studies, interested readers can find examples for German texts (Petran, 2012) and Swedish texts (Bouma and Adesam, 2013).

Huang et al.


(2010) employed the techniques of conditional random fields (CRFs) to segment texts of literature and history. They achieved 0.7899 and 0.9179 in F
₁


,


³

respectively, for segmenting the texts in Shiji and Zuozhuan.


⁴

Wang et al. (2016; 2017) applied recurrent neural networks to segment texts in a diverse collection of classical Chinese sources. They achieved F
₁ measures that are close to 0.75, and item accuracies that are near 0.91.


⁵

The researchers achieved different segmentation results for different corpora even when they applied the same techniques and procedures. It is thus inappropriate to just compare the numbers for ranking because the nature of the corpora varies widely.

In this proposal, we report our attempts to segment texts in tomb biographies with CRF models (Lafferty, 2001). We studied the effects of considering different types of lexical information in the models, and achieved 0.853 in precision, 0.807 in recall, 0.829 in F
₁, and 0.940 in item accuracy


.


⁶

Better results were accomplished when we employed deep learning techniques, including applicaitons of long short-term memory networks and sequence-to-sequence networks, for segmenting our tomb biographies.

2.
Data Sources

We obtained digitized texts for three books of tomb biographies of the Tang dynasty (Zhou and Zhao, 1992; 2001). The collection consists of 5119 biographies which contain 423,922 periods, commas, and semicolons. There are 5505 distinct types of characters and a total of more than 2461 thousand of characters in the collection.


⁷

When counting these statistics, we ignored a very small portion of characters that cannot be shown without special fonts. Hence, these statistics are not perfectly precise, but they are accurate within a reasonable range. On average, a biography has about 480 characters. Some of them are very short and have many missing characters. Hence, we exclude biographies that have no more than 30 characters in our experiments.


⁸

3.
Training and Testing CRF Models

We consider the segmentation task as a classification problem. Let C
_i denote an individual character in the texts. We categorize each character as either
M (for “followed by a punctuation mark”) and
O (for “an ordinary character”). Assume that we should add only a punctuation mark between C
₃ and C
₄ for a string “C
₁ C
₂ C
₃ C
₄ C
₅”. A correct labeling for this string will be “O O M O O”.


⁹

We can convert each character in the texts into an
instance, which may be used for training or testing. We provide with each instance a group of contextual
features that may be relevant to the judgment of whether or not a punctuation mark is needed. For instance, we may use one character surrounding a character X and itself as the group of features. The following are two instances that we create for C
₃ and C
₄. The instance for C
₃ is (1), and the leftmost item is the correct label for C
₃, and the rest are the features for C
₃.


¹⁰

M w[0]=C
₃,w[-1]=C
₂,w[1]=C
₄ (1)

O w[0]=C
₄,w[-1]=C
₃,w[1]=C
₅ (2)

We can train a CRF model with a selected portion of the instances (called
training data), and test the resulting model with the remaining instances (called
test data). The instances in the training and the test data are mutually exclusive.

We employ a machine-learning tool that learns from the training data to build a CRF model.


¹¹

We then apply the learned model to predict the classes of the instances in the test data. The labels of the instances in the test data are temporarily concealed when the learned model makes predictions.


¹²

The precision rate and recall rate of the learned model are calculated with the correct and the predicted labels.

We report four sets of basic experiments next, each investigating an important aspect for analyzing Chinese texts. The 5119 biographies were resampled and randomly assigned to the training (70%) and test (30%) sets for every experiment.


¹³

We repeated every experiment three times, and report the averages of the precision and recall rates.

3.1.
Changing the Size of the Context

We certainly can and should consider more than one character around the current character as the context. Figure 2 shows the test results of using different sizes of contexts for the instances. The horizontal axis shows the sizes, e.g., when
k=2, the feature set includes information about two characters on both sides of the current character. P1 and R1 are the average precision and recall rates, respectively.

Figure 2. Effects of varying context sizes

We expected to improve the precision and recall rates by expanding the width of the context. The margin of improvements gradually decreased, and the curves level off after the window sizes reached six. The recall rises sharply when we add the immediate neighbor word into the features, emphasizing the predicting power of the immediate neighbor character. When
k=10, the precision and recall are 0.765 and 0.729, respectively, and the item accuracy exceeds 0.91.

3.2.
Adding Bigrams

We added bigrams that were formed by consecutive characters into the features. The following instance shows the result of adding bigrams to the features in (1).


¹⁴

M w[0]=C
₃,w[-1]=C
₂,w[1]=C
₄,
w[-1_0]=C
₂
C
₃
,w[0_1]=C
₃
C
₄ (3)

Figure 3 shows the test results of adding bigrams while we also tried different sizes of context. The curves named P1 and R1 are from Figure 2, and P2 and R2 are results achieved by adding bigrams to the features. Both rates are improved, and the gains are remarkable.

Figure 3. Adding bigrams improves the results.

4.
Discussions

We have consulted historians,
<sup>6,


18</sup>

and learned that our current results are useful in practice. The best precision rates and F measures are better than 0.8 in Figure 3 and Table 2. The best item accuracy is better than 0.94.

In fact, we have designed an advanced mechanism to further improve our results.


¹⁹

The new approach employs a second level learning step that learns from the errors of the current classifiers.

One may plan to consider more linguistic information in the segmentation tasks. If appropriate corpora or sources are available, it is worthwhile to explore the effects of adding part-of-speech information in the task (Chiu, 2015; Lee, 2012). We have applied deep learning techniques for the segmentation tasks, and achieved better results.

Although we look for methods to reproduce the segmentations in the given texts, we understand that not all experts will agree upon “the” segmentations for a corpus. Different segmentations may correspond to different interpretations of the texts, especially for the classical Chinese. The results of asking two persons to segment Chinese texts may not match perfectly either (Huang and Chen, 2011).